Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

Export Folding is not UTF-8 Safe

#11

The folding which happens when exporting a calendar currently splits at a fixed character position.

This may cause a UTF-8 string with multi-byte characters to be split producing invalid UTF-8 data.

Reported by Rick DeNatale · May 27th, 2009 @ 11:38 AM

State: resolved
Milestone: none
Assigned to: Rick DeNatale Rick DeNatale

Activity

  1. Fjan
    Fjan

    Very clever. You only checked the remainder and it worried me that there could be a UTF-8 character that would split into something that was still valid, but I checked the UTF-8 spec and that doesn't seem possible. In fact, you can also determine if you are dealing with a split-in-the-middle character by simply checking if the top two bits are '10', which may be a bit cheaper than String.unpack

    May 28th, 2009 @ 07:55 AM

  2. Rick DeNatale
    Rick DeNatale
    • State changed from new to resolved

    I suspect that String.unpack is actually quicker than checking the byte with ruby, since it's a C 'primitive'

    In any case I doubt that it makes much performance difference, I trust String#pack to get the utf-8 rules right.

    May 28th, 2009 @ 08:34 AM

  3. Fjan
    Fjan

    By the way, here is what I came up with, slightly shorter but a bit less elegant:

    class String
    def ical_wrap if (l=self.length)>75 then x=75 begin if self[x]&192!=128 then self.insert(x,"\r\n ") # change for Ruby 1.9 elsif self[x-=1]&192!=128 then self.insert(x,"\r\n ") else self.insert(x-=1,"\r\n ") end x+=77;l+=3 end while x<l end self end end

    May 28th, 2009 @ 01:32 PM

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.